Skip to content

Add support for local LLM - #133

Open
foureight84 wants to merge 1 commit into
saksham2001:mainfrom
foureight84:feat/local-llm-coach
Open

Add support for local LLM#133
foureight84 wants to merge 1 commit into
saksham2001:mainfrom
foureight84:feat/local-llm-coach

Conversation

@foureight84

Copy link
Copy Markdown
Contributor

Adds a localOpenAICompat provider pointing at any OpenAI-Chat-Completions server the user runs themselves — Ollama, llama.cpp, vLLM, SGLang, LM Studio — so a coach conversation can happen without health data leaving the network. Ported from PulseLoopAndroid #51 including the fixes from its review.

Structurally the client is MiniMaxClient — same Responses→Chat translation, same accumulate-across-send statefulness, same fresh-client-per-turn contract — with four differences, each forced by something local backends do:

  • The API key is optional. Every engine in scope runs unauthenticated by default, so a blank key omits the Authorization header and readiness is a base URL that passes LocalEndpoint.validate. Not "non-empty": Settings persists as the user types, so a blank-check would flip the coach to "Active" on the first character and fail every turn afterwards.
  • developer folds into system, and all system turns merge into one leading message. SGLang 400s on a role outside its pydantic Literal; many local chat templates need the system turn first and singular.
  • Capabilities are declared, not assumed. vLLM 400s on tools without --enable-auto-tool-choice; LM Studio has no json_object mode. Detect probes a plain baseline request first, then tools and response_format, so a 4xx means "this field is refused" rather than "this request is refused" — and only a conclusive verdict overwrites a setting the user chose by hand.
  • A long, configurable timeout, and no redirects.

Privacy note, per CONTRIBUTING: this changes what leaves the device only by making it possible for nothing to. Cleartext is three layers. NSAllowsLocalNetworking re-permits cleartext for local destinations only (never NSAllowsArbitraryLoads), LocalEndpoint.validate explains a bad host in the Settings field instead of failing opaquely at request time, and LocalHTTP refuses redirects — validate() vets the URL typed, not where the request lands, and a 307 off the LAN would resend the health-context body in the clear.

New under Coach/Local/: LocalEndpoint, LocalHTTP, LocalModelCatalog, LocalCapabilityProbe, LocalOpenAICompatClient, LocalLLMKeychainStore, and LocalServerSettingsSection (its own view — the probe state should die with the provider selection, and CoachSettingsSection was at SwiftLint's type_body_length ceiling). The target uses a file-system-synchronized group, so no project.pbxproj change.

Docs: design and the per-engine support matrix in docs/local-llm-coach.md (added to the mkdocs nav); README, docs/platforms/ios-vs-android.md and docs/project/privacy.md updated to list the provider.

swiftlint: 0 errors. 1108 tests, 0 failures (43 new). Not yet verified against a real server.

Summary

Related issues

Type of change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 📟 New / improved wearable support (BLE driver layer)
  • 🤖 Coach / LLM change (tools, prompts, orchestration)
  • 🎨 UI / DesignSystem change
  • 🧹 Refactor / chore (no behavior change)
  • 📝 Docs only
  • ⚠️ Breaking change (existing data, settings, or APIs change)

How was this tested?

  • Added / updated unit tests (PulseLoopTests)
  • Ran the test suite locally (⌘U in Xcode)
  • Tested on a physical device with a real ring — model:
  • Tested with demo data (-seedDemo YES, no hardware)
  • N/A (docs / non-code change)

Privacy & data

  • This change does not send health data off-device without explicit user action.
  • No secrets, API keys, or personal data are committed.
  • N/A

Screenshots / recordings

Checklist

  • My code follows the project's style (SwiftLint passes).
  • I ran the tests and they pass.
  • I updated docs / README where relevant.
  • I read the Contributing guide.

Adds a `localOpenAICompat` provider pointing at any OpenAI-Chat-Completions
server the user runs themselves — Ollama, llama.cpp, vLLM, SGLang, LM Studio
— so a coach conversation can happen without health data leaving the network.
Ported from PulseLoopAndroid saksham2001#51 including the fixes from its review.

Structurally the client is MiniMaxClient — same Responses→Chat translation,
same accumulate-across-send statefulness, same fresh-client-per-turn contract
— with four differences, each forced by something local backends do:

- The API key is optional. Every engine in scope runs unauthenticated by
  default, so a blank key omits the Authorization header and readiness is a
  base URL that passes LocalEndpoint.validate. Not "non-empty": Settings
  persists as the user types, so a blank-check would flip the coach to
  "Active" on the first character and fail every turn afterwards.
- `developer` folds into `system`, and all system turns merge into one leading
  message. SGLang 400s on a role outside its pydantic Literal; many local chat
  templates need the system turn first and singular.
- Capabilities are declared, not assumed. vLLM 400s on `tools` without
  --enable-auto-tool-choice; LM Studio has no json_object mode. Detect probes
  a plain baseline request first, then tools and response_format, so a 4xx
  means "this field is refused" rather than "this request is refused" — and
  only a conclusive verdict overwrites a setting the user chose by hand.
- A long, configurable timeout, and no redirects.

Privacy note, per CONTRIBUTING: this changes what leaves the device only by
making it possible for nothing to. Cleartext is three layers.
NSAllowsLocalNetworking re-permits cleartext for local destinations only
(never NSAllowsArbitraryLoads), LocalEndpoint.validate explains a bad host in
the Settings field instead of failing opaquely at request time, and LocalHTTP
refuses redirects — validate() vets the URL typed, not where the request
lands, and a 307 off the LAN would resend the health-context body in the clear.

New under Coach/Local/: LocalEndpoint, LocalHTTP, LocalModelCatalog,
LocalCapabilityProbe, LocalOpenAICompatClient, LocalLLMKeychainStore, and
LocalServerSettingsSection (its own view — the probe state should die with the
provider selection, and CoachSettingsSection was at SwiftLint's
type_body_length ceiling). The target uses a file-system-synchronized group,
so no project.pbxproj change.

Docs: design and the per-engine support matrix in docs/local-llm-coach.md
(added to the mkdocs nav); README, docs/platforms/ios-vs-android.md and
docs/project/privacy.md updated to list the provider.

swiftlint: 0 errors. 1108 tests, 0 failures (43 new).
Not yet verified against a real server.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant